Mechanism for Manager and Host-Based Integrated Power Saving Policy in Virtualization Systems

ABSTRACT

A mechanism for a manager and host-based integrated power saving policy in virtualization systems is disclosed. A method of the invention includes receiving configuration and power information of a host machine from a management agent on the host machine, performing a macro-level power saving scheduling algorithm that takes into consideration the received configuration and power information of the host machine, and requesting that the host machine alter a number of active running CPU cores as part of the macro-level power saving scheduling algorithm.

TECHNICAL FIELD

The embodiments of the invention relate generally to virtualizationsystems and, more specifically, relate to a mechanism for a manager andhost-based integrated power saving policy in virtualization systems.

BACKGROUND

In computer science, a virtual machine (VM) is a portion of softwarethat, when executed on appropriate hardware, creates an environmentallowing the virtualization of an actual physical computer system. EachVM may function as a self-contained platform, running its own operatingsystem (OS) and software applications (processes). Typically, a virtualmachine monitor (VMM) manages allocation and virtualization of computerresources and performs context switching, as may be necessary, to cyclebetween various VMs.

A host machine (e.g., computer or server) is typically enabled tosimultaneously run multiple VMs, where each VM may be used by a local orremote client. The host machine allocates a certain amount of the host'sresources to each of the VMs. Each VM is then able to use the allocatedresources to execute applications, including operating systems known asguest operating systems. The VMM virtualizes the underlying hardware ofthe host machine or emulates hardware devices, making the use of the VMtransparent to the guest operating system or the remote client that usesthe VM.

One goal of the above-described virtualization systems is to reducepower consumption in the system with a power saving strategy. Currently,there are two mechanisms to control power consumption in avirtualization system: (1) a macro-level power saving policy implementedby a central manager such as a host controller, and (2) a micro-levelpower saving policy implemented by each host machine. The macro-levelpower saving policy includes a central manager trying to consolidateworkload on a small number of host machines, so that other host machinescan be shutdown. One example of a macro-level power saving schedulingpolicy is to reduce power utilization by consolidating the workload on asmall number of host machines and shutting down host machines that arenot used in order to reduce overall power consumption. The centralmanager is usually slower in making changes (e.g., minutes), as itresponds to trends.

The micro-level power saving policy is implemented at a local hostmachine scheduler, which shuts down unused local resources (e.g., CPUcores) in order to save power one each individual host machine. Thelocal manager is usually faster in responding to changes (e.g., secondsand less) than a central manager of the macro-level power saving policy.

However, there can be more to a power saving policy than shutting down asingle host machine. Some power saving policies include the ability toshut down individual components of a host machine, rather than the hostmachine itself. For example, a power saving policy can shut down CPUcores, network interface cards (NICs), and so on. This is where amicro-level power saving policy comes in, which can throttle powerwithin a single host machine by shutting down individual componentswithin the machine. Unfortunately, there is no collaboration betweenmacro-level power saving policies and micro-level power saving policies.This can lead to inefficiencies in overall power consumption in avirtualization system.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the invention. The drawings, however, should not be takento limit the invention to the specific embodiments, but are forexplanation and understanding only.

FIG. 1 is a block diagram of an exemplary virtualization architecture inwhich embodiments of the present invention may operate;

FIG. 2 is a flow diagram illustrating a method performed by a hostcontroller for manager and host-based integrated power saving policy invirtualization systems according to an embodiment of the invention;

FIG. 3 is a flow diagram illustrating a method performed by a hostmachine for manager and host-based integrated power saving policy invirtualization systems according to an embodiment of the invention; and

FIG. 4 illustrates a block diagram of one embodiment of a computersystem.

DETAILED DESCRIPTION

Embodiments of the invention provide a mechanism for a manager andhost-based integrated power saving policy in virtualization systems. Amethod of embodiments of the invention includes receiving configurationand power information of a host machine from a management agent on thehost machine, performing a macro-level power saving scheduling algorithmthat takes into consideration the received configuration and powerinformation of the host machine, and requesting that the host machinealter a number of active running CPU cores as part of the macro-levelpower saving scheduling algorithm.

In the following description, numerous details are set forth. It will beapparent, however, to one skilled in the art, that the present inventionmay be practiced without these specific details. In some instances,well-known structures and devices are shown in block diagram form,rather than in detail, in order to avoid obscuring the presentinvention.

Some portions of the detailed descriptions which follow are presented interms of algorithms and symbolic representations of operations on databits within a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise, as apparent from the followingdiscussion, it is appreciated that throughout the description,discussions utilizing terms such as “sending”, “receiving”, “attaching”,“forwarding”, “caching”, or the like, refer to the action and processesof a computer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (electronic)quantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions, each coupled to a computer system bus.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear as set forth in thedescription below. In addition, the present invention is not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the invention as described herein.

The present invention may be provided as a computer program product, orsoftware, that may include a machine-readable medium having storedthereon instructions, which may be used to program a computer system (orother electronic devices) to perform a process according to the presentinvention. A machine-readable medium includes any mechanism for storingor transmitting information in a form readable by a machine (e.g., acomputer). For example, a machine-readable (e.g., computer-readable)medium includes a machine (e.g., a computer) readable storage medium(e.g., read only memory (“ROM”), random access memory (“RAM”), magneticdisk storage media, optical storage media, flash memory devices, etc.),a machine (e.g., computer) readable transmission medium (non-propagatingelectrical, optical, or acoustical signals), etc.

Embodiments of the invention provide a mechanism for a manager andhost-based integrated power saving policy in virtualization systems. Theintegrated power saving policy of embodiments of the inventioncollaborate between (1) a system-wide power saving policy implemented ata host controller machine, and (2) a per-host machine locallyimplemented power saving policy. The host controller machine is aware ofeach individual host machine's power saving policy and takes thisinformation into consideration when implementing a system-wide powersaving policy.

FIG. 1 illustrates an exemplary virtualization architecture 100 in whichembodiments of the present invention may operate. The virtualizationarchitecture 100 may include one or more host machines 110 to run one ormore virtual machines (VMs) 112. Each VM 112 runs a guest operatingsystem (OS) that may be different from one another. The guest OS mayinclude Microsoft Windows, Linux, Solaris, Mac OS, etc. The host machine110 may include a hypervisor 115 that emulates the underlying hardwareplatform for the VMs 112. The hypervisor 115 may also be known as avirtual machine monitor (VMM), a kernel-based hypervisor or a hostoperating system.

In one embodiment, each VM 112 may be accessed by one or more of theclients over a network (not shown). The network may be a private network(e.g., a local area network (LAN), wide area network (WAN), intranet,etc.) or a public network (e.g., the Internet). In some embodiments, theclients may be hosted directly by the host machine 110 as a localclient. In one scenario, the VM 112 provides a virtual desktop for theclient.

As illustrated, the host 110 may be coupled to a host controller 105(via a network or directly). In some embodiments, the host controller105 may reside on a designated computer system (e.g., a server computer,a desktop computer, etc.) or be part of the host machine 110 or anothermachine. The VMs 112 can be managed by the host controller 105, whichmay add a VM, delete a VM, balance the load on the server cluster,provide directory service to the VMs 112, and perform other managementfunctions.

The host controller 105 includes a power policy agent 107 thatimplements the integrated power saving policy of embodiments of theinvention. Power policy agent 107 of host controller 105 implements amacro-level power saving scheduling policy in conjunction with loadbalancing and migration agent 109. Additionally, in embodiments of theinvention, power policy agent 107 is aware of a host machine 110micro-level power saving scheduling policy and utilizes this awarenessto aid in the host controller's 105 own macro-level power saving policydecisions.

With respect to the local host machine 110 micro-level power savingscheduling policy, this micro-level power saving policy operates so thateach host machine 110 may change any of its cores' power states. In onecase, a core's power state is altered by reducing the core's speed orshutting the core down when load on the host machine 110 is less than X%for over Y milliseconds. In another case, a core's power state ischanged by raising its speed or activating the core when load on thehost machine 110 is more than X% for over Y milliseconds. For both ofthe above calculations, the X% is based on total CPU utilization dividedby the number of active cores (i.e., average distributed CPUconsumption), then dividing this result by 100 and dividing again by thenumber of total cores on the host machine (i.e., portion in percentageof CPU affected by a single core). In addition, for both of the abovecalculations, a power state change that affects a speed change is basedon the ratio between the pre and post speeds, multiplied by the core CPUpercentage portion. In addition, the local scheduler may use a notationprovided by the host controller 105 on lower priority tasks and pin themto cores with lower speeds.

The host controller 105 macro-level power saving scheduling policy takesseveral approaches to making decisions on consolidating workload toreduce power saving policy. Each of these approaches is augmented inembodiments of the invention with the knowledge of the host machine's110 micro-level power saving policy. Specifically, in embodiments of theinvention, the power policy agent 107 communicates with a managementagent 125 of hypervisor 115 on each host machine 110 in order todetermine configuration and power policy information for each hostmachine 110. This configuration and power policy information includesthe total number of CPU cores as well as the number of active CPU coreson any given host machine 110 that the host controller 105 manages.Furthermore, the management agent 125 may provide information on theactual power consumption of each core on the host machine 110. In someembodiments, this configuration and power policy information iscollected every several seconds from the management agent 125, whichmonitors and collects this information by communication with alower-level power saving scheduling daemon in the host machine 110.

When the host controller 105 is performing any load balancing schedulingchecks, the CPU consumption for a service level agreement (SLA) on ‘max’(distribute to other hosts) and ‘min’ (consolidate to other hosts) isbased on the total number of cores in the host machine 110, rather thanthe active number of cores. For example, assume a host machine 110 isutilizing 80% load overall. However, the host machine 110 is running the80% load only on 2 running cores, but actually the host machine 110 has10 more cores that could be activated. In this situation, embodiments ofthe invention will allow the host controller 105 to take the informationon the total number of cores in the host machine 110 into account inorder to avoid activating another host machine 110, when it can justactivate more cores on the same host machine 110 that is alreadyrunning.

Similarly, when the host controller 105 is performing any host selectioncomparison algorithms, the host controller 105 may take into account thegranularity of activating another core on the host machine 110, giving ahigher score to host machines 110 where an expected load (based onhistory) will require the least amount of cores to be activated(expected).

In some embodiments, the host controller 105 may take into account thepower consumption per core on the host machine 110 (different hosts havedifferent power consumptions per core). In one embodiment, the hostcontroller 105 may request the local scheduler 130 of a host machine 110to activate more cores if it deems the SLA policy is not met beforemigrating load to another host machine 110. The host controller 105 mayalso request the local scheduler 130 of a host machine 110 to shut downmore cores if it deems the power saving policy goals are not met. Thiscould be tied, for example, to different costs associates withelectricity over different hours of the day. This will be done if thehost controller 105 could not suspend any more tasks, but can allow themto run in a lower priority state and take longer.

FIG. 2 is a flow diagram illustrating a method 200 performed by a hostcontroller for manager and host-based integrated power saving policy invirtualization systems according to an embodiment of the invention.Method 200 may be performed by processing logic that may comprisehardware (e.g., circuitry, dedicated logic, programmable logic,microcode, etc.), software (such as instructions run on a processingdevice), or a combination thereof. In one embodiment, method 200 isperformed by host controller 105, and, more specifically, the powerpolicy agent 107 of host controller 105 described with respect to FIG.1.

Method 200 begins at block 210 where configuration and power informationrelated to a host machine are received from a management agent locatedon that host machine. In one embodiment, the configuration informationincludes the total number of cores available on the host machine as wellas the number of active running cores on the host machine. The powerinformation may include the actual power consumption of each core on thehost machine.

Then, at block 220, a macro-level power saving scheduling algorithm isperformed as part of load balancing or a host selection process. Themacro-level power saving scheduling algorithm takes into account thereceived configuration and power information of the individual hostmachine. For instance, the macro-level power saving scheduling algorithmwill look at the total number of cores available on the host machine,rather than just the number of active running cores, when determine howto distribute load (i.e., VMs) or schedule load between host machines.In one embodiment, the macro-level scheduler looks at the total cores onthe host machine, as it schedules based on the potential use of allcores on the host machine. The micro-level scheduler looks at the activenumber of cores, and if the average CPU utilization is above a certainService Level Agreement (SLA), it will activate another core (as long asthe number of active cores is less than or equal to the number of totalcores, of course).

At block 230, as part of the macro-level power saving schedulingalgorithm, the host controller requests the host machine to alter itstotal number of active running cores. In one embodiment, this may meanrequesting the host machine to active more cores in order to distributeadditional load across the host machine rather than activating anotherhost machine. In other embodiments, this may mean requesting the hostmachine to shut down cores in order to consolidate load, and reducepower consumption, on the host machine.

At block 240, the host controller may also optionally denote to the hostmachine loads that have a lower priority so that the denoted loads mayrun on cores on the host machine that have a lower speed. In this case,the host controller relied on the power information received at block210 to make this determination. Lastly, at block 250, the hostcontroller schedules one or more VMs away from or to the host machinebased on the results of the macro-level power saving schedulingalgorithm.

FIG. 3 is a flow diagram illustrating a method 300 performed by a hostmachine for manager and host-based integrated power saving policy invirtualization systems according to an embodiment of the invention.Method 300 may be performed by processing logic that may comprisehardware (e.g., circuitry, dedicated logic, programmable logic,microcode, etc.), software (such as instructions run on a processingdevice), or a combination thereof. In one embodiment, method 300 isperformed by host machine 110 of FIG. 1.

Method 300 begins at block 310 where configuration and power informationare sent to a host controller. In one embodiment, this configuration andpower information may be collected by a power saving scheduling daemonand provided to a management agent of a hypervisor in the host machine.The management agent, in turn, sends this information to a hostcontroller that manages the host machine. At block 320, a request toalter the number of active running cores in the host machine isreceived. In one embodiment, the request may be to active one or morecores in the host machine. In another embodiment, the request may be toshut down one or more cores in the host machine.

Subsequently, at block 330, the number of running cores on the hostmachine is altered according to the request. Lastly, at block 340,scheduling instructions are received at the host machine to at least oneof receive or migration one or more VMs at the host machine as part of amacro-level power saving scheduling algorithm performed by the hostcontroller.

FIG. 4 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system 400 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeembodiments, the machine may be connected (e.g., networked) to othermachines in a LAN, an intranet, an extranet, or the Internet. Themachine may operate in the capacity of a server or a client machine in aclient-server network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be apersonal computer (PC), a tablet PC, a set-top box (STB), a PersonalDigital Assistant (PDA), a cellular telephone, a web appliance, aserver, a network router, switch or bridge, or any machine capable ofexecuting a set of instructions (sequential or otherwise) that specifyactions to be taken by that machine. Further, while only a singlemachine is illustrated, the term “machine” shall also be taken toinclude any collection of machines that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein.

The exemplary computer system 400 includes a processing device 402, amain memory 404 (e.g., read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) (such as synchronous DRAM (SDRAM) or RambusDRAM (RDRAM), etc.), a static memory 406 (e.g., flash memory, staticrandom access memory (SRAM), etc.), and a data storage device 418, whichcommunicate with each other via a bus 430.

Processing device 402 represents one or more general-purpose processingdevices such as a microprocessor, central processing unit, or the like.More particularly, the processing device may be complex instruction setcomputing (CISC) microprocessor, reduced instruction set computer (RISC)microprocessor, very long instruction word (VLIW) microprocessor, orprocessor implementing other instruction sets, or processorsimplementing a combination of instruction sets. Processing device 402may also be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. The processing device 402 is configured to execute theprocessing logic 426 for performing the operations and steps discussedherein.

The computer system 400 may further include a network interface device408. The computer system 400 also may include a video display unit 410(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), analphanumeric input device 412 (e.g., a keyboard), a cursor controldevice 414 (e.g., a mouse), and a signal generation device 416 (e.g., aspeaker).

The data storage device 418 may include a machine-accessible storagemedium 428 on which is stored one or more set of instructions (e.g.,software 422) embodying any one or more of the methodologies offunctions described herein. For example, software 322 may storeinstructions to perform a manager and host-based integrated power savingpolicy in virtualization systems by host controller 105 described withrespect to FIG. 1. The software 422 may also reside, completely or atleast partially, within the main memory 404 and/or within the processingdevice 402 during execution thereof by the computer system 400; the mainmemory 404 and the processing device 402 also constitutingmachine-accessible storage media. The software 422 may further betransmitted or received over a network 420 via the network interfacedevice 408.

The machine-readable storage medium 428 may also be used to storedinstructions to perform manager and host-based integrated power savingpolicy in virtualization systems of methods 200 and 300 described withrespect to FIGS. 2 and 3, and/or a software library containing methodsthat call the above applications. While the machine-accessible storagemedium 428 is shown in an exemplary embodiment to be a single medium,the term “machine-accessible storage medium” should be taken to includea single medium or multiple media (e.g., a centralized or distributeddatabase, and/or associated caches and servers) that store the one ormore sets of instructions. The term “machine-accessible storage medium”shall also be taken to include any medium that is capable of storing,encoding or carrying a set of instruction for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present invention. The term “machine-accessiblestorage medium” shall accordingly be taken to include, but not belimited to, solid-state memories, and optical and magnetic media.

Whereas many alterations and modifications of the present invention willno doubt become apparent to a person of ordinary skill in the art afterhaving read the foregoing description, it is to be understood that anyparticular embodiment shown and described by way of illustration is inno way intended to be considered limiting. Therefore, references todetails of various embodiments are not intended to limit the scope ofthe claims, which in themselves recite only those features regarded asthe invention.

What is claimed is:
 1. A computer-implemented method, comprising:receiving, by a host controller machine, configuration and powerinformation of a host machine from a management agent on the hostmachine; performing, by the host controller machine, a macro-level powersaving scheduling algorithm that takes into consideration the receivedconfiguration and power information of the host machine; and requesting,by the host controller machine, that the host machine alter a number ofactive running CPU cores as part of the macro-level power savingscheduling algorithm.
 2. The method of claim 1, wherein theconfiguration information includes a total number of CPU cores on thehost machine and the number of active running CPU cores on the hostmachine.
 3. The method of claim 2, wherein taking into consideration thereceived configuration and power information of the host machine furthercomprises considering the total number of CPU cores when at least one ofload balancing the host machine or considering the host machine as partof a host selection algorithm.
 4. The method of claim 3, furthercomprising denoting one or more loads as having a lower priority to thehost machine so that the denoted one or more loads run on CPU coreshaving a lower power consumption.
 5. The method of claim 1, wherein thepower information includes an actual power consumption of each CPU coreon the host machine.
 6. The method of claim 1, wherein the requestingthat the host machine alter a number of active running CPU cores furthercomprises requesting the host machine to active one or more of itsinactive CPU cores.
 7. The method of claim 1, wherein the requestingthat the host machine alter a number of active running CPU cores furthercomprises requesting the host machine to shut down one or more of itsactive running CPU cores.
 8. The method of claim 1, further comprisingscheduling one or more virtual machines (VMs) to the host machine basedon the macro-level power saving scheduling algorithm.
 9. A system,comprising: a memory; a processor communicably coupled to the memory;and a power policy agent executed from the memory by the processor, thepower policy agent configured to: receive configuration and powerinformation of a host machine from a management agent on the hostmachine; perform a macro-level power saving scheduling algorithm thattakes into consideration the received configuration and powerinformation of the host machine; and request that the host machine altera number of active running CPU cores as part of the macro-level powersaving scheduling algorithm.
 10. The system of claim 9, wherein theconfiguration information includes a total number of CPU cores on thehost machine and the number of active running CPU cores on the hostmachine.
 11. The system of claim 10, wherein taking into considerationthe received configuration and power information of the host machinefurther comprises considering the total number of CPU cores when atleast one of load balancing the host machine or considering the hostmachine as part of a host selection algorithm.
 12. The system of claim11, wherein the power policy agent further configured to denote one ormore loads as having a lower priority to the host machine so that thedenoted one or more loads run on CPU cores having a lower powerconsumption.
 13. The system of claim 9, wherein the power informationincludes an actual power consumption of each CPU core on the hostmachine.
 14. The system of claim 9, wherein the requesting that the hostmachine alter a number of active running CPU cores further comprises atleast one of requesting the host machine to active one or more of itsinactive CPU cores or requesting the host machine to shut down one ormore of its active running CPU cores.
 15. The system of claim 9, furthercomprising scheduling one or more virtual machines (VMs) to the hostmachine based on the macro-level power saving scheduling algorithm. 16.An article of manufacture comprising a machine-readable storage mediumincluding data that, when accessed by a machine, cause the machine toperform operations comprising: receiving configuration and powerinformation of a host machine from a management agent on the hostmachine; performing a macro-level power saving scheduling algorithm thattakes into consideration the received configuration and powerinformation of the host machine; and requesting that the host machinealter a number of active running CPU cores as part of the macro-levelpower saving scheduling algorithm.
 17. The article of manufacture ofclaim 16, wherein the configuration information includes a total numberof CPU cores on the host machine and the number of active running CPUcores on the host machine.
 18. The article of manufacture of claim 17,wherein taking into consideration the received configuration and powerinformation of the host machine further comprises considering the totalnumber of CPU cores when at least one of load balancing the host machineor considering the host machine as part of a host selection algorithm.19. The article of manufacture of claim 18, wherein the machine-readablestorage medium includes data that, when accessed by the machine, causethe machine to perform further operations comprising denoting one ormore loads as having a lower priority to the host machine so that thedenoted one or more loads run on CPU cores having a lower powerconsumption.
 20. The article of manufacture of claim 16, wherein therequesting that the host machine alter a number of active running CPUcores further comprises at least one of requesting the host machine toactive one or more of its inactive CPU cores or requesting the hostmachine to shut down one or more of its active running CPU cores.